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I. REAL PARTY IN INTEREST 

The real party of interest in the present application is the assignee of the 
present application, Xerox Corporation. 
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II. RELATED APPEALS AND INTERFERENCES 

There is no related appeal or interference. 
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III. STATUS OF CLAIMS 

Claims 1-22 are on appeal. 

Claims 1-22 are pending in this application. Of these, claims 1,11, and 12 are 
independent claims. 

Claims 1-22 have been finally rejected in an Office Action mailed October 6, 
2005 (hereinafter referred to as the "Final Office Action") and Advisory Action mailed 
December 20, 2005 (hereinafter referred to as the "Advisory Action"), on the grounds 
further discussed herein. 
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IV. STATUS OF AMENDMENTS 

No amendments were filed subsequent to the final rejection of Claims 1-22 in 
the Final Office Action. All amendments made to the claims prior to the Final Office 
Action are therefore believed to be entered. 
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V. SUMMARY OF CLAIMED SUBJECT MATTER 

Generally, Appellant's invention recited in each independent claim 1,11, and 
12 is directed at a system and method that enables users at different locations to 
work collaboratively on a hardcopy document, and to view each other's annotations 
to the hardcopy document. As shown in Appellant's Figure 1 below, the system 10 
includes a plurality of workstations 14 communicating with a server 12. Each 
workstation 14 comprises a capture device 20, such as a camera, a computer 
processor 16, and a display unit 18 for capturing and displaying a digital image of 
the hardcopy document 22 with handwritten annotations. (See Appellant's 
specification on page 4, lines 9-11.) Handwritten annotations are identified in a 
captured image of the hardcopy document 22 at each workstation 14. (See 
Appellant's specification on page 4, lines 23-30.) Data representing the identified 
annotation images is communicated to each workstation to permit an annotation 
entered at a first workstation 14 to the hardcopy document 22- and an annotation 
entered at a second workstation to the hardcopy document 22 to be distributed to 
the plurality of workstations 14. (See Appellant's specification on page 5, lines 20- 
22.) The data representing the identified annotations is used for display with the 
digital images of the hardcopy document 22 at the plurality of workstations according 
to display criteria for each workstation such that one or more of the annotations are 
selectively displayed in accordance with the display criteria of each workstation 14. 
(See Appellant's specification on page 5, lines 23-27.) 




FIG. I 
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In addition, Appellant's invention recited in dependent claims 7 and 18 
(together with the claims 4 and 15, respectively, to which they depend) provides that 
the data representing each identified annotation is stored in the form of an 
annotation object, where the annotation object includes: (a) information representing 
the annotation as a bitmap; (b) information indicating the location of the annotation 
in a digital image of the hardcopy document; and (c) information representing the 
identity of the digital image of the hardcopy document. (See Appellant's specification 
on page 5, lines 9-15.) 

Appellant's Figures 4(a)-4(c) illustrate three handwritten annotations 48a, 
48b, and 48c made by three different people and Figure 5 illustrates how the 
annotations 48a, 48b, and 48c in Figures 4(a)-4(c) may be combined together for 
collaborative viewing. (See Appellant's specification on page 6, lines 1-4.) 
Advantageously, with the claimed system and method the user is advantageously 
able to work directly on hardcopy documents and not limited only to making 
electronic annotations. (See Appellant's specification on page 4, lines 18-21.) 




animals in: because it organism, already 




FIG. 5 
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VI. GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

Claims 1-6, 8-17, and 19-22 stand rejected under 35 U.S.C. §1 03(a) as being 
unpatentable over Carleton et al., U.S. Patent No. 5,781,727 (hereinafter referred to 
as "Carleton") in view of Iran, U.S. Patent No. 6,054,990 and further in view of 
Simonoff , U.S. Patent No. 6,351,777 (hereinafter referred to as "Simonoff"), further 
in view of Cass , U.S. Patent No. 5,692,073 (hereinafter referred to as "Cass"). 

Claims 7 and 18 stand rejected under 35 U.S.C. §1 03(a) as being 
unpatentable over Carleton in view of Tran and Simonoff and further in view of 
Levine et al., U.S. Patent No. 5,680,636 (hereinafter referred to as "Levine"), further 
in view of Cass. 
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VII. ARGUMENT 

Appellant respectfully traverses the rejection of the pending claims and 
submits they are in condition for allowance for the reasons set forth below. 

A. First Group Of Claims, Consisting Of Claims 1 (And Its Dependent Claims 2- 
6, 8-10, and 21), 11, and 12 (And Its Dependent Claims 12-17, 19-20, and 22), Is 
Patentable Over Carleton, Tran, Simonoff, and Cass 

In this section, Appellant traverses the rejection of the first group of claims, 
consisting of independent claims 1 (and its dependent claims 2-6, 8-10, and 21), 11, 
and 12 (and its dependent claims 12-17, 19-20, and 22) as being obvious under 35 
U.S.C. §1 03(a) over Carleton in view of Tran, Simonoff, and Cass. The instant 
rejection with regard to independent claim 1, which is discussed herein as the 
representative claim of independent claims 11 and 12, is set forth on pages 2-4 of 
the Final Office Action. 

Appellant respectfully submits that Carleton taken singly or in combination 
with Tran, Simonoff, and/or Cass fails to disclose or suggest claim 1 read as a 
whole which includes identifying handwritten annotations made to a hardcopy 
document captured at a first and a second workstations, which annotations are 
communicated to each workstation where one or more of the annotations are 
selectively displayed in accordance with display criteria of each workstation. 

Carleton describes a system which permits electronic annotations to an 
application program made by users at remote computers to appear on the displays 
of other participants. More specifically, Carleton describes a system where an 
application program can run on a single computer, yet remote participants can issue 
commands to the program. Further, participants can make annotations on their 
displays, which are copied by the system onto the displays of other participants. 
(See Abstract of Carleton.) 

For example, Figure 10 of Carlton, reproduced below, illustrates a remote 
user annotating a calculator and that annotation appearing on other remote user or 
host displays. (See Carleton column 7, lines 43-67.) 
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FIG. 10 

MODE: ANNOTATION 

,- - - REMOTES DRAW BOX ALSO DRAWS BOX 




REMOTE HOST REMOTE 

PROGRAM IS RUNNING HERE 



Unlike Appellant's claimed invention which provides a system that allows 
users at different locations to collaboratively share annotations made to a hardcopy 
document, Carleton discloses a system for running an application program on one 
display where electronic annotations of it may be shared on other displays (see 
Appellant's Figures 1, 4, and 5 reproduced above in the Summary Of Claimed 
Subject Matter). 

Tran discloses "a graphical data entry system for accepting and processing 
hand sketches and writings such that the user can quickly specify graphical objects 
in a drawing on a hand-held, mobile computer with a relatively compact screen". 
(See Abstract of Tran.) "The graphical data entry system [ ] can be used in 
conjunction with a camera to annotate pictures taken with the camera". (See 
Abstract of Tran.) In addition, pen strokes are entered to the system using a pen 
sensed by a digitizer unit via an electromagnetic field. (See Tran column 7, lines 47- 
52.) 

Figure 13 of Tran, reproduced below, and the following text set forth at 
column 20, lines 7-20 and lines 24-37 describes how annotations of recorded 
images are to be annotated: 

In step 240, the routine retrieves camera settings from the photographic 
instruments and, if the instrument settings are to be annotated on the film, the 
routine formats the data in preparation for encoding. Next, the routine 
determines if data has been entered via the keypad or buttons/switches 24. If so, 
the routine stores the typed text data in step 242 before it proceeds to step 249 
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to encode the data. Alternatively, if data has not been entered via the 
keypad/switches 24, the routine checks if verbal annotations have been entered 
in step 243. If so, the routine performs a speech to text conversion in step 244 
before proceeding to step 249. 

Alternatively, from step 243, if no verbal annotations have been entered, 
the routine checks if pen annotations have been entered in step 245. If so, the 
routine proceeds to step 248 to convert the handwritten annotations into text 
using the hand recognition process discussed above. From step 248, the routine 
jumps to step 249 to format and encode the data. From step 245, if the 
handwritten annotations are not available, the routine checks for sketches as 
annotations. If the pen stroke is a part of a sketch, the routine proceeds with the 
draw routine of FIG. 5 in step 247. From step 242, 244, 247 or 248, the routine 
formats and provides the cumulated data to the data encoder to encode the 
camera settings and user annotations in step 249 before the routine of FIG. 13 is 
exited. 




FIG. 13 
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Thus, unlike Appellant's claimed invention which provides a system that 
allows users at different locations to collaboratively share annotations made to a 
hardcopy document, Tran discloses a system where images captured with a camera 
may then be annotated using keyboard annotations, verbal annotations, and 
annotations made with a pen/digitizer unit that are either converted to text or 
considered a sketch. 

Simonoff discloses "a dedicated White Board system [that facilitates] 
collaboration between a plurality of users". Objects and hyperlinks and text may be 
placed and tracked on the white board with tools. (See Simonoff abstract.) A white 
board applet is run on a client host computer connects with the white board 
application server (see Simonoff column 10, lines 1-13). A user may draw objects on 
the white board and share them according to a set of permissions (see Simionoff 
column 1 1 , line 52 to column 12, line 4). For example, as shown in Simonoff's Figure 
3, reproduced below, white board applet 301a on client hosts 300a and 300b 
connects to white board application server 102 running on server 100 (see Simionoff 
column 10, lines 1-13). 

FIG. 3 







/1 03 


101^ 










^102 




Unlike Appellant's claimed invention which provides a system that allows 
users at different locations to collaboratively share annotations made to a hardcopy 
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document, Simonoff discloses a system where annotations are shared that are 
made to an electronic white board. 

Cass discloses a method for using arbitrary documents as computer readable 
forms, where any kind of stored digital document may be used as a form (i.e., 
"formless forms"). In the method illustrated in Cass Figure 5 reproduced below, a 
reference document to which a mark has been added is extracted therefrom. 
Information is provided concerning a set of active elements of the reference 
document, where the reference document has at least one active element 
associated with at least one action. The extracted mark is evaluated to determine 
whether it indicates any of the active elements, which if one is indicated the action 
associated therewith is performed. (See Cass column 3, lines 23-59.) [Note, the 
Final Office Action cites lines 12-19 of Cass but not the column number - Appellant 
has assumed for the purpose of the discussion herein that the Final Office Action on 
page 4, line 2 refers to column 3 of Cass.] 




FIG. 5 



Unlike Appellant's claimed invention which provides a system that allows 
users at different locations to collaboratively share annotations made to a hardcopy 
document, Cass discloses a method for using flexible hardcopy forms by using an 



12 



Application No. 09/738,992 



active element(s) that is associated with a reference document to which a mark has 
been extracted from it. 

The forgoing discussion of the cited references illustrate that combining 
Carlton, Tran, Simonoff, and Cass taken singly or together, fails to disclose or 
suggest a system as claimed by Appellant in claim 1 in which handwritten 
annotations made to a hardcopy document captured at a first and a second 
workstations are identified, which annotations are communicated to each 
workstation where one or more of the annotations are selectively displayed in 
accordance with display criteria of each workstation. 

Moreover, Carlton, Tran, and Simonoff taken together or singly concern the 
annotation of an electronic document; none disclose or suggest the sharing of 
annotations made to a hardcopy document; for example: Carlton allows participants 
to create and share annotations to an application program on an electronic display 
(see Carlton Abstract); Tran permits the annotation of pictures taken with a camera 
using a graphical data entry system (see Tran Figure 13 discussed above); and 
Simonoff allows users to draw and share objects on an electronic white board (see 
Simonoff column 1 1 , line 51 to column 12, line 5). 

Cass in contrast discloses a method for using flexible hardcopy forms, which 
method relies on at least one active element(s) that is associated with a reference 
document to which a mark has been extracted from it. When the mark and the 
action element are associated, the action corresponding to the element is 
performed. In the PaperWeb example described in Cass (see Cass column 15, line 
51 through column 19, line 6), hypertext links in paper documents may be followed 
by specifying a link as an action element and where the action of a marked link is to 
return the document associated with the action element. 

Appellant respectfully submits that Carlton, Tran, and Simonoff taken singly or 
together with Cass fail to disclose or suggest the element of Appellant's claimed 
invention as recited in independent claim 1 when taken as a whole, which recites a 
system where handwritten annotations made to a hardcopy document captured at a 
plurality of workstations are identified, and which annotations are communicated to 
each workstation wherein one or more of the annotations are selectively displayed in 
accordance with display criteria of each workstation, as Cass concerns the use of 
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active elements in paper-based forms and not the combination of annotations made 
by a plurality of users to a document. 

That is for the reasons discussed above, Carlton, Tran, Simonoff, and/or 
Cass read singly or together, fail to disclose or suggest a system that permits users 
at a plurality of workstations to selectively view handwritten annotations identified in 
digital images of a hardcopy document captured at each workstation with a capture 
device, as claimed by Appellant. Further for the reasons discussed above, the cited 
references fail to disclose or suggest a system and method as claimed by Appellant 
that allows users at different locations to work collaboratively on a hardcopy 
document by allowing annotations to a hardcopy document to be viewed at a 
plurality of workstation displays in accordance with the display criteria of each 
workstation, as the claimed system and method provide that annotations made to 
the hardcopy document are communicated between them. 

Accordingly, Appellant respectfully submits for the reasons set forth above 
that independent claim 1 is patentably distinguishable over Carlton, Tran, Simonoff, 
and/or Cass as the cited sections in the references fail to disclose or suggest 
Appellant's claimed invention which recites: capturing a digital image of a hardcopy 
documents at a plurality of workstations; identifying handwritten annotations in the 
captured images; communicating data representing the handwritten annotations to 
the plurality of workstations; displaying the hardcopy document at the workstations 
while selectively displaying the annotations in accordance with the display criteria of 
each workstation. 

In addition it should be noted that claims 11 and 12 contain the same or very 
similar limitations to those discussed above with respect to claim 1 , and therefore 
the argument presented above with regard to claim 1 applies equally to claims 1 1 
and 12. Insofar as claims 2-6, 8-10 and 13-17, and 19-22 are concerned, these 
claims depend from presumably allowable independent claims 1 and 12 and are 
also therefore believed to be patentably distinguishable over Carlton, Tran, 
Simonoff, and/or Cass for those reasons set forth for claims 1 and 12. 

B. Second Group Of Claims, Consisting Of Claims 7 and 18, Is Patentable Over 
Carleton, Tran, Simonoff, Cass, and Levine 

In this section, Appellant traverses the rejection of the second group of 
claims, consisting of dependent claims 7 and 18 as being obvious under 35 U.S.C. 
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§1 03(a) over Carleton in view of Tran, Simonoff, Cass, and Levine. The instant 
rejection with regard to claims 7 (which depends from claims 4, which in turn 
depends from claim 1) and 18 (which depends from claim 15, which in turn depends 
from claim 12) is set forth on pages 13-14 (and page 10 with regard to claims 4 and 
15) of the Final Office Action. 

Appellant respectfully submits that Carleton taken singly or in combination 
with Tran, Simonoff, Cass, and/or Levine fails to disclose or suggest claims 7 and 
18 read as a whole which provides that the data representing each identified 
annotation stored in the form of an annotation object in the system recited in claim 1 
(and its corresponding method recited in claim 12) includes: (a) information 
representing the annotation as a bitmap, (b) information indicating the location of the 
annotation in a digital image of the hardcopy document, and (c) information 
representing the identity of the digital image of the hardcopy document with which 
the annotation is associated. 

In contrast, Carleton, Tran, Simonoff, and Cass described subject matter as 
discussed above with respect to claim 1 , while Levine discloses a system adapted 
to display an annotated bitmap image of a document. The system has two views, a 
desk view and an annotation session view. The desk view emulates a user's desk, 
whereas the annotation view provides screen size document views of a document to 
be annotated or created using an input device, such as, a stylus or pen. (See Levine 
column 2 line 59 to column 3 line 45.) 

Unlike Appellant's claimed invention which provides a system that allows 
users at different locations to collaboratively share annotations made to a hardcopy 
document, Levine discloses a system where annotations are made using a stylus (or 
pen) to a bitmap and that such annotations may be superimposed on a bitmap 
image (see Levine Abstract and column 7, lines 1-8). More specifically, any 
association or cataloguing of annotations performed by Levine (see Levine column 
7, lines 1-8), such association or cataloguing fails to concern information 
representing the identity of a digital image of a hardcopy document as claimed by 
Appellant. 

Accordingly, Appellant respectfully submits for the reasons set forth above 
that claims 7 and 18 are patentably distinguishable over Carlton, Tran, Simonoff, 
Cass, and/or Levine as the cited sections in the references fail to disclose or suggest 
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Appellant's claimed invention which recites in part that data representing each 
annotation includes (a) information representing the annotation as a bitmap, (b) 
information indicating the location of the annotation in a digital image of the 
hardcopy document, and (c) information representing the identity of the digital image 
of the hardcopy document with which the annotation is associated. 

C. Conclusion 

Based on the arguments presented above, claims 1-22 are believed to be in 
condition for allowance. Appellant therefore respectfully requests that the Board of 
Patent Appeals and Interferences reconsider this application, reverse in whole the 
rejection of claims 1-22, and pass this application for allowance. 



Respectfully submitted, 




Thomas Zell 
Attorney for Appellant 
Registration No. 37,481 
Telephone: 650-812-4281 



Date: 6/6/2006 
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CLAIMS APPENDIX 

CLAIMS INVOLVED IN THE APPEAL: 

1 . A system for permitting collaborative annotation of a hardcopy document, 
the system comprising: 

a plurality of workstations each comprising a computer processor, a display, 
and a capture device for capturing a digital image of a hardcopy document; and 

a base computer communicating with the plurality of workstations; 

wherein the system is operative to: 

(a) identify handwritten annotations in digital images of the hardcopy 
document captured at each workstation; 

(b) communicate data representing the identified annotation images to each 
workstation to permit an annotation entered at a first workstation to the hardcopy 
document and an annotation entered at a second workstation to the hardcopy 
document to be distributed to the plurality of workstations; 

(c) use the data representing the identified annotations for display with the 
digital images of the hardcopy document at the plurality of workstations according to 
display criteria for each workstation; and 

(d) display the digital images of the hardcopy document at the workstations 
with one or more of the identified annotations entered at the first and second 
workstations; 

wherein the one or more of the annotations are selectively displayed in 
accordance with the display criteria for each workstation. 

2. The system according to claim 1 , wherein the system is operative to 
identify a new annotation made at a third workstation or an additional annotation 
made by the first or second workstations, and to update the distribution of 
annotations to the plurality of workstations. 

3. The system according to claim 1 , wherein the capture device at, at least 
one, workstation comprises a camera. 
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4. The system according to claim 1 , wherein the data representing each 
identified annotation is stored in the form of an annotation object, the annotation 
object comprising: 

(a) information representing the annotation as a bitmap; and 

(b) information indicating the location of the annotation in a digital image of 
the hardcopy document. 

5. The system according to claim 4, wherein the annotation object further 
comprises information representing the origin of the annotation. 

6. The system according to claim 4, wherein the annotation object further 
comprises information representing at least one of a time and date associated with 
the annotation. 

7. The system according to claim 4, wherein the annotation object further 
comprises information representing the identity of the digital image of the hardcopy 
document with which the annotation is associated. 

8. The system according to claim 5, wherein at least one of the plurality of 
workstations is operative to display annotations in a plurality of different colors 
according to its display criteria, the color being indicative of the origin of the 
annotation. 

9. The system according to claim 1 , wherein each workstation is operative to 
identify handwritten annotations. 

10. The system according to claim 1 , wherein the base computer is operative 
to identify handwritten annotations. 

1 1 . A method for distributing annotations to a hardcopy document made at a 
plurality of workstation of a collaborative annotation system to other workstations in 
the system for display, comprising: 

(a) capturing a digital image of the hardcopy document at each workstation; 

(b) identifying a portion of the digital image corresponding to a handwritten 
annotation in the hardcopy document captured at a first workstation and a second 
workstation; 
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(c) distributing data representing the handwritten annotation to each 
workstation to permit the annotation entered at the first workstation to the hardcopy 
document and the annotation entered at a second workstation to the hardcopy 
document to be distributed to the plurality of workstations; 

(d) using the data representing the identified annotations for display with the 
digital images of the hardcopy document at the plurality of workstations according to 
display criteria for each workstation; and 

(e) displaying the digital images of the hardcopy document at the workstations 
with one or more of the identified annotations entered at the first and second 
workstations; 

wherein the one or more of the annotations are selectively displayed in 
accordance with the display criteria for each workstation; and 

wherein the one or more of the annotations are selectable at each workstation 
for retrieving information associated with each selectively displayed annotation, 
including information concerning each annotation's creation and author. 

12. A method for permitting collaborative annotation of a hardcopy document 
in a system which comprises: 

a plurality of workstations each comprising a computer processor, a display, 
and a capture device for capturing a digital image of a hardcopy document; and 

a base computer communicating with the plurality of workstations; 

wherein the method comprises: 

(a) identifying handwritten annotations in digital images of the hardcopy 
document captured at each workstation; 

(b) communicating data representing the identified annotation images to each 
workstation to permit an annotation entered at a first workstation to the hardcopy 
document and an annotation entered at a second workstation to the hardcopy 
document to be distributed to the plurality of workstations; and 

(c) using the data representing the identified annotations for display with the 
digital images of the hardcopy document at the plurality of workstations according to 
display criteria for each workstation; 
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(d) displaying the digital images of the hardcopy document at the workstations 
with one or more of the identified annotations entered at the first and second 
workstations; 

wherein the one or more of the annotations are selectively displayed in 
accordance with the display criteria for each workstation. 

13. The method according to claim 12, further comprising identifying a new 
annotation made at a third workstation or an additional annotation made by the first 
or second workstations, and updating the distribution of annotations to the plurality 
of workstations. 

14. The method according to claim 12, further comprising capturing an image 
at, at least one, workstation using a camera device. 

15. The method according to claim 12, wherein each identified annotation is 
stored as annotation objects comprising: 

(a) information representing the annotation as a bitmap; and 

(b) information indicating the location of the annotation in a digital image of 
the hardcopy document. 

16. The method according to claim 15, wherein the annotation object further 
comprises information representing the origin of the annotation. 

17. The method according to claim 15, wherein the annotation object further 
comprises information representing at least one of a time and date associated with 
each identified annotation. 

18. The method according to claim 15, wherein the annotation object further 
comprises information representing the identity of the digital image of the hardcopy 
document with which the annotation is associated. 

19. The method according to claim 12, further comprising selectively 
displaying, at a third workstation, a plurality of annotations originating from at least 
the first and second workstations. 

20. The method according to claim 19, wherein each annotation is selectively 
displayed in a color indicative of the origin of the annotation according to the display 
criteria. 
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21 . The system according to claim 1 , wherein the one or more of the 
annotations are selectable at each workstation for retrieving information associated 
with each selectively displayed annotation, including information concerning each 
annotation's creation and author. 

22. The method according to claim 12, wherein the one or more of the 
annotations are selectable at each workstation for retrieving information associated 
with each selectively displayed annotation, including information concerning each 
annotation's creation and author. 
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